Due to the increasing amount of video data, a lot of research has been done in the field of retrieving and categorizing this type of data. On the other hand, with the growing popularity of football and the increasing number of its audiences, the importance of automatic and real-time extraction of statistics and information about Soccer matches has increased. One of the critical and challenging tasks in Soccer video analysis is the detection of players’ blobs and regions, along with identifying the teams related to the players. This task encounters many challenges, including grass loss in the playfield, the presence of playfield lines and players' shadows, the overlapping of players with objects and other players, and different shapes of players in different situations. This paper proposes a framework for detecting players and their related teams. For this purpose, an object-sieve-based method is used to detect players’ blobs, and a genetic algorithm is used to identify their related teams. Each chromosome of the genetic algorithm is a window that lies on one blob whose fitness function shows how much its color and shape characteristics fit with the uniforms of each of the two teams. The proposed method was evaluated by 50 different frames of broadcast Soccer videos, including 563 players, and 40 different sub-images, including 84 players. The results show 98% and 91.6% precision for player detection and labeling, respectively.